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Abstract 

Background: Given that patients provide the majority of their own diabetes care, patient self-management training 
has increasingly become recognized as an important strategy with which to improve quality of care. However, 
participation in self management programs is low. In addition, the efficacy of current behavioural interventions 
wanes over time, reducing the impact of self-management interventions on patient health. Web-based 
interventions have the potential to bridge the gaps in diabetes care and self-management. 

Methods: Our objective is to improve self-efficacy, quality of life, self-care, blood pressure, cholesterol and glycemic 
control and promote exercise in people with type 2 diabetes through the rigorous development and use of a 
web-based patient self-management intervention. This study consists of five phases: (1) intervention development; 
(2) feasibility testing; (3) usability testing; (4) intervention refinement; and (5) intervention evaluation using mixed 
methods. We will employ evidence-based strategies and tools, using a theoretical framework of self-efficacy, then 
elicit user feedback through focus groups and individual user testing sessions. Using iterative redesign the 
intervention will be refined. Once finalized, the impact of the website on patient self-efficacy, quality of life, self- 
care, HbA1c, LDL-cholesterol, blood pressure and weight will be assessed through a non-randomized observational 
cohort study using repeated measures modeling and individual interviews. 

Discussion: Increasing use of the World Wide Web by consumers for health information and ongoing revolutions 
in social media are strong indicators that users are primed to welcome a new era of technology in health care. 
However, their full potential is hindered by limited knowledge regarding their effectiveness, poor usability, and high 
attrition rates. Our development and research agenda aims to address these limitations by improving usability, 
identifying characteristics associated with website use and attrition, and developing strategies to sustain patient use 
in order to maximize clinical outcomes. 
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development, Study protocol, User-Computer Interface, Repeated measures modeling, Qualitative methods 



* Correspondence: yuca@smh.ca 

1 Keenan Research Centre, Li Ka Shing Knowledge Institute, St. Michael's 
Hospital, 30 Bond Street, Toronto, ON M5B 1 W8, Canada 
department of Medicine, University of Toronto, Toronto, Canada 
Full list of author information is available at the end of the article 

O© 201 2 Yu et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative 
BlOlVlGCl C^ntrBl Commons Attribution License (http://creativecommons.Org/licenses/by/2.0), which permits unrestricted use, distribution, and 
reproduction in any medium, provided the original work is properly cited. 



Yu et al. BMC Medical Informatics and Decision Making 2012, 12:57 
http://www.biomedcentral.com/1472-6947/12/57 



Page 2 of 10 



Background 

Management of diabetes is complex, and involves con- 
trolling the multiple risk factors that lead to complica- 
tions. Given that patients provide the majority of their 
own diabetes care [1], patient self-management training 
is an important strategy with which to improve quality 
of care [1]. Several systematic reviews have examined 
the impact of self-management interventions on gly- 
cemic control, cardiac risk factors and psychological out- 
comes. These reviews have demonstrated positive effects 
on knowledge [2], self- reported dietary habits [2], quality 
of life [3], and glycemic control [2,4]. However, effects of 
interventions on lipids, physical activity, weight, and 
blood pressure were variable [2]. Characteristics of ef- 
fective interventions include patient collaboration and 
regular reinforcement [2]. 

Despite this sound rationale and evidence supporting 
these interventions, participation in self management 
programs is low [5]. In addition, the efficacy of behav- 
ioural interventions wanes over time [6], reducing the 
impact of self-management interventions on patient 
health. Consumers are increasingly accessing the World 
Wide Web and social media as sources of health infor- 
mation [7,8]. Web-based interventions have the potential 
to improve diabetes care and self-management. Two sys- 
tematic reviews suggest that web-based media improve 
knowledge and understanding [9,10], social support [10], 
behaviour change [10] and clinical outcomes [10] for a 
variety of diseases. However effective education and self- 
management principles have not been systematically 
incorporated into existing diabetes websites for patients 
to optimize these interventions. A review of existing dia- 
betes websites demonstrated that these websites were 
characterized by didactic information of variable quality, 
high reading levels, and a newspaper-format display, 
with little interactive technology, social support or 
problem-solving assistance [11,12]. A recent systematic 
review of electronic diabetes-related tools found that 
they had moderate but inconsistent effects on a variety 
of psychological and clinical outcomes including HbAlc 
and weight and a high prevalence of usability errors [13]. 
It also found that more interactive tools resulted in con- 
tinued website use and greater clinical improvement. 

Thus, we sought to improve effective self-manage- 
ment, as evidenced by improved self-efficacy, quality of 
life, self-care, blood pressure, cholesterol and glycemic 
control and exercise promotion, in people with type 2 
diabetes through the use of a web-based patient self- 
management intervention that addresses the identified 
limitations of existing web-based interventions. Specific- 
ally, we sought to ensure inclusion of evidence-based 
content in a usable format, with incorporation of inter- 
activity, social support and problem-solving assistance, 
patient collaboration and regular reinforcement. 



Methods 

Intervention development requires careful planning and 
the use of theory-based strategies to increase the prob- 
ability of effectiveness, programme adoption and imple- 
mentation [14]. Graham et al's Knowledge to Action 
framework [15] was chosen for evaluating this know- 
ledge translation activity (Table 1) because it is context- 
focused, enables knowledge-producer and knowledge- 
user collaboration, and emphasizes sustainability. The 
Knowledge to Action framework incorporates the need 
to adapt the knowledge to fit with the local context, 
which is particularly important for this activity: this pro- 
ject is occurring in the context of the provincial govern- 
ment's 5-year diabetes initiative, which includes a web- 
based dissemination strategy. Secondly, this framework 
is particularly useful for emphasizing the collaboration 
between knowledge producers and knowledge users 
throughout the process and for facilitating the use of re- 
search knowledge by several stakeholders, such as prac- 
titioners, policymakers, patients and the public. By 
involving stakeholders in the choice of question and 
interventions, the external validity of the research is 
strengthened. Finally, it addresses the need to sustain 
knowledge use by anticipating changes and adapting ac- 
cordingly. As described above, sustainable knowledge 
use is essential given the chronic nature of diabetes. In 
this paper, we describe our innovative systematic ap- 
proach to intervention development and evaluation. 

Study overview 

This study consists of five phases: (1) intervention devel- 
opment; (2) feasibility testing; (3) usability testing; (4) 
intervention refinement; and (5) intervention evaluation 
using mixed methods (repeated measure modeling, indi- 
vidual interviews). 

Phase 1: Intervention development 

Our objective was to create an evidence-based, theory- 
driven self-contained website. Based on our systematic 
review of electronic tools on topics relevant to diabetes 
[13], we selected tools that were known to be effective, 
relevant and usable. For example, we included multi- 
media diabetes education modules targeting individuals 
with low literacy that were demonstrated to be effective 
in lowering Ale [16] and that were easy to use [13]. In 
addition, we incorporated behavioral intervention strat- 
egies that were found to be effective in a systematic re- 
view of internet-based interventions, including stress 
management and communication tools [17]. For ex- 
ample, we included text- and graphic-based material on 
coping with diabetes, an online forum to communicate 
with peers, as well as tips and website-generated reports 
to facilitate communication with health care providers. 
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Table 1 Knowledge-to-Action framework 



Knowledge to Action framework 
[15] 



Study phase 



Identify problem. Identify, review, 
select knowledge 



Adapt knowledge to local context 



Assess barriers to knowledge use 



Select, tailor, implement 
interventions 



Monitor knowledge use 
Evaluate outcomes 
Sustain knowledge use 



Patient self-management training has increasingly become recognized as an important strategy with which to 
narrow the care gap. Several systematic reviews have examined the impact of diverse self-management 
interventions and have demonstrated positive effects on knowledge self-reported dietary habits [6], quality of 
life [3] and glycemic control [4,6]. Despite the strength of this evidence base, participation in these programs 
is low. Given the growing prevalence of diabetes worldwide as well as the strong evidence base upon which 
available guidelines were developed, effectively bridging the knowledge to practice gap in this area has the 
potential to significantly improve health care outcomes and thus health care delivery and system 
sustainability. 

h July 2008 the Ontario MOHLTC launched the Ontario Diabetes Strategy, to improve prevention and care for 
Ontarians with chronic diseases, starting with diabetes, through a mix of prevention, access to technology, 
personal planning and access to specialized resources and health professionals. All Ontarians with diabetes 
and their health care providers will be supported through a series of inter-related initiatives. As part of this 
larger initiative, there is an implementation plan for patient self-management tools, as well as a plan for 
measuring and reporting on improvements in clinical care and outcomes on a web-based patient portal. 

Barriers to knowledge use can occur at several levels, including the health care system, the health care team 
and organization, the health care profession, the patient and, finally, the guidelines or their education delivery 
system. Brown described a similar framework, and categorized barriers to diabetes care on three levels: 
organization, provider, and patient. Barriers at the patient level include acceptance of the diagnosis, 
education, self-motivation and adaptation to daily living. Poor adherence to guidelines may be a result of 
patient preferences, expectations or knowledge. Our initiative will focus on barriers at the patient level. 

(a) Selecting the intervention (Phase 1)Web-based interventions have the potential to bridge the gaps in 
diabetes care and self-management. Two systematic reviews suggest that web-based media improve 
knowledge or understanding [9,10], social support [10], behaviour change [10] and clinical outcomes [10] in a 
variety of disease states, (b) Tailoring the intervention. The intervention will be refined following feasibility 
(Phase 2) and usability testing (Phase 3) with patients with diabetes, (c) Implementing the intervention. The 
refined self-management tool (Phase 4) will be implemented in a pilot study (Phase 5). 

A mixed method study, as described in the main text, will be conducted, consisting of an interrupted time 
series and individual interviews (Phase 5). 

Barriers to sustained knowledge use will be addressed in the planning phase of tool development, 
implementation and dissemination, and further explored upon completion of the pilot study with qualitative 
methodology (Phase 5). 



We integrated these interventions with the theoretical 
framework of self-efficacy, a theory that has not only 
been validated in predicting and promoting patient be- 
havior change but has also been demonstrated to im- 
prove clinical outcomes [18-25]. For example, diabetes 
self-management education programs incorporating self- 
efficacy have been shown in randomized controlled trials 
to improve knowledge [25], health behaviour [24,25], 
self-efficacy [23-25], HbAlc [23-25], and weight [25] and 
microvascular complications [24]. Briefly, self-efficacy 
refers to "beliefs in one's capabilities to organize and 
execute the courses of action required to produce given 
attainments" [26]. It arises from one's successes or fail- 
ures during previous performances, observations of 
others' experiences, verbal persuasion and physiological 
and affective states, and is mediated by cognitive, motiv- 
ational, affective and selective processes. We used these 
sources and mediators of self-efficacy in formatting our 
site and selecting tools. For example, we incorporated 
feedback, goal-setting, peer story-telling, and monitoring 
tools into the website by including computer-generated 
responses to user entries, a goal-setting application, vid- 
eos of peer testimonials, trackers for blood glucose, 



blood pressure, weight and physical activity and a diary) 
(Figure 1). 

We also drew from the literature insights regarding 
patient health information seeking behavior to promote 
optimal use of the website [27]. Developed through 
qualitative focus group methodology, the approach of 
Longo et al. acknowledges the role of passive receipt and 
active retrieval of information, as well as the role of rela- 
tionships with other individuals and health care provi- 
ders in helping to process and interpret information. We 
used this model of health information seeking behavior 
to inform the development of interfaces that would en- 
able a flexible approach to information seeking. To this 
end, we will automate email briefs of selected content, 
optimize search algorithms to enable self-directed re- 
trieval, create an online forum to permit sharing of 
experiences, and develop tools (e.g. blood pressure track- 
ers) and content (e.g. "How to prepare for your appoint- 
ment with your physician") to facilitate communication 
with health care providers (Figure 1). 

We chose to create a self-contained website that did 
not require regular one-on-one input by a health care 
provider to optimize sustainability. 
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Theory 



Website features 



Evidence 





Health information model 
(theory of health information - 
seeking behaviour)(27) 
•Active information seeking 
•Passive information receiving 
•Health literacy 



Self-efficacy 

(theory of behaviour change)(26) 
•Enactive mastery experience 
•Physical/affective state 
•Cognitive processes 
•Motivational processes 
•Selection processes 
•Verbal persuasion 




Search/browse function 

Email notification 
Educational comic strip 
Video-based peer story- 
telling 
Online community 

Goal-setting 
Things I do well 
Blood glucose tracker 
Blood pressure tracker 
Food & activity tracker 
Web-diary 
Email prompts 
Stress management tools 

Communication tools 
Knowledgeable, credible 
source 




Systematic review of diabetes- 
related electronic tools(1 3): 
•Living with diabetes(16) 
•My Blood Pressure Action 
Plan 

•Stop Smoking Centre 



Systematic review of 
behaviourchange 
websites(17): 

•Multiple behaviourchange 
tools 

•Stress management tools 
Communications skills 



Figure 1 Evidence and theory-based framework for intervention development. Schematic depicting theoretical underpinnings (health 
information model, self-efficacy) and evidence base (systematic reviews of electronic tools, behavior change websites) contributing to website 
features. 



Phase 2: Feasibility testing 

We will employ focus group methodology to assess ac- 
ceptability, usability, sustainability, strengths and weak- 
nesses of this intervention. The website will be presented 
to participants who will be asked to work through the web 
based tools to complete a task that simulates real clinical 
use. The task invites participants to use the website to de- 
termine their personal risk of heart disease and specific 
strategies to reduce this risk, including the role of blood 
pressure in vascular risk and methods to reduce their 
blood pressure. 

Participants 

A purposive sampling strategy will be used to generate a 
heterogeneous group that captures the perspectives of pa- 
tient participants with varied experiences. Participants 
(age greater than 25 years) with varied socio-demographic 
profiles (e.g. gender, age, use of insulin, ethnicity, educa- 
tional attainment, annual income) and technological profi- 
ciency will be recruited from diabetes care centers at two 
academic health science centers in Toronto, Canada, and 
will complete a baseline questionnaire of user demograph- 
ics. A sample size of 3 focus groups is estimated to achieve 
saturation of themes and to generate sufficient feedback 
to refine the tool; while sample size of 5-8 people is 



generally required for a homogeneous group, we wanted 
to sample a heterogeneous group as outlined above [28] . 

Outcomes 

We will probe about intervention content, such as web- 
site title, general website organization, website searching 
tools, format of interactive and didactic tools, and con- 
tent that they would find valuable. We are also inter- 
ested in intervention processes, such as barriers and 
facilitators to the adoption of the specific tools and of 
the website generally, factors affecting continued use, 
reasons to visit the site repeatedly, the role of social net- 
working, and comfort with entering personal informa- 
tion online. We want to determine features of the 
components that are perceived by participants to be 
valuable, and those which are in need of further 
development. 

Data collection 

Focus group sessions will employ semi-structured inter- 
view guides developed by team members with know- 
ledge translation and qualitative research expertise [29]. 
Face and content validity of this guide will be assessed 
with team members and refined as needed. All focus 
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group interviews will be digitally audio-recorded and 
field notes kept. 

Data analysis 

Audiotapes will be transcribed verbatim [30]. In keeping 
with qualitative methodology, data analysis will occur in 
conjunction with data collection [31]. Transcripts will be 
coded for emergent categories and themes using con- 
stant comparison [31]. All transcripts will be coded and 
reviewed independently by three team members with ex- 
pertise in qualitative research methods; consensus on 
coding will be reached through comparison, discussion, 
and agreement among these three reviewers. Further, 
memos kept by the qualitative team members will be 
used to help monitor the emergent analytic framework 
and regular meetings will enable continued dialogue and 
discussion within the project [32]. NVivo software (ver- 
sion 9) will be used to assist data management and 
monitor the emerging analysis. 

Phase 3: Usability testing 

We will conduct usability testing sessions using cogni- 
tive task analysis [33], whereby users are individually 
asked to "think aloud" as they perform specific tasks 
representing scenarios designed to cover the major func- 
tionalities of the web-based intervention in a one-hour 
session. We plan to do additional cycles of usability test- 
ing after intervention refinement in an iterative process 
of evaluation and redesign. 

Participants 

A consecutive sample of ten participants with type 2 dia- 
betes aged greater than 25 years will be recruited from 
diabetes care centers at two academic health science 
centers in Toronto, Canada. The sample size should be 
sufficient, as research has shown that up to 80% of us- 
ability issues can be identified with 5 to 8 participants 
[33]. 

Outcomes 

Data regarding paths users take to accomplish tasks, 
errors made, when and where they encountered confu- 
sion or frustration, time spent, and degree of satisfaction 
will be generated. 

Data collection 

A consultant with expertise in health informatics and 
human factors engineering will conduct each session, 
and further prompt users with a structured interview 
guide on usability. All usability testing sessions will be 
video and audio-taped. In addition, field notes will be 
kept of all sessions as a further source of data for ana- 
lysis. Video tapes of the computer screen will be used to 
capture user actions in response to prompted tasks. 



Data analysis 

Audio tapes will be transcribed verbatim. Transcripts 
will be coded for emergent categories and themes using 
constant comparison [31]. Transcripts will be coded and 
reviewed independently by three team members with ex- 
pertise in qualitative research methods, and consensus 
on the coding framework will be reached [34,35]. NVivo 
software (version 9) will be used to assist with data man- 
agement and retrieval for analysis. Video tapes will be 
used in conjunction with audio tapes to identify the area 
of the website in discussion and user actions recorded. 

Phase 4: Intervention refinement 

Based on data from the feasibility testing, refinements 
will be made to the website via ongoing discussion with 
the research and development team. Similarly, following 
usability testing, further refinements will be made to the 
website in an iterative process of testing and redesign. 
We anticipate that refinement will occur over a one- 
month period. 

Phase 5: Intervention evaluation 

We hypothesize that a web-based patient self-management 
intervention, developed and revised based on our usability 
and feasibility testing, will result in improvements in self- 
care score, self-efficacy score, health care user satisfaction 
score and quality of life score in patients with type 2 dia- 
betes with access to the internet. 

Observational cohort study 

Participants 

A consecutive sample of individuals aged > 25 years with 
hemoglobin Ale (HbAlc) > 7.0%, systolic blood pressure 
(sBP) > 130 mmHg, low-density-lipoprotein cholesterol 
(LDL-C) > 2.0 mmol/L or a body mass index (BMI) > 25 kg/ 
m 2 will be recruited from 2 family practice units and 2 
endocrinology clinics at 2 urban academic health science 
centers in Toronto. Those with active heart disease (defined 
as Canadian Cardiovascular Society class 3 to 4 angina), 
who were non-English-speaking, who were not available for 
follow-up, or who had no regular access to the telephone 
or internet will be excluded. Age, gender, ethnicity, educa- 
tion, employment, duration of diabetes, complications, 
smoking status, anti-hyperglycemic, antihypertensive, and 
lipid-lowering agents, HbAlc, sBP, LDL-C, weight, current 
use and comfort with computer and internet, self-care 
score, self-efficacy score, health care user satisfaction score 
and quality of life score will be obtained at baseline. 

Outcomes 

Primary outcome 

The primary outcome, self efficacy, will be assessed with 
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the Modified Grossman Self-efficacy for Diabetes Scale 
which has moderate to high reliability (Cronbach's 
alpha = 0.51 to 0.86) [36,37] (Table 2). 

Secondary outcomes 

Secondary outcomes include self care and quality of life, 
which will be assessed through well-validated scales, 
described below and in Table 2. 

Self-care behavior will be assessed with the Summary 
of Diabetes Self-Care Activities Measure - Revised. Reli- 
ability and validity for this instrument have been 
reported to be fair, with a test-retest correlation of 0.40 
and internal consistency of 0.47 [38,39]. 

Diabetes-specific quality of life will be assessed with 
the Diabetes Distress Scale which has been found to 
have high internal reliability with a Cronbach's alpha of 
0.93, good convergent validity with the Center for Epi- 
demiological Studies Depression Scale (CESD) (r = 0.56) 
and self-care behaviors including lower adherence to eat- 
ing recommendations (r = 0.30, p < 0.001) and lower 
levels of physical activity (r = 0.13, p<0.01) [40]. In 
addition, diabetes distress has been demonstrated to be 
associated with HbAlc (r = 0.17-0.31, P= .00-.001), diet 
(r = -0.38, P=.00), physical activity (r=-0.13, P = M) 
and medication adherence (r= -0.16, P= .00) [41,42]. 

These patient-based outcomes were selected because 
they are direct and relevant measures of knowledge use 
by patients. Diabetes self-care is a direct measure of 
knowledge use, and self-efficacy was chosen in order to 



better understand the mediating variables of knowledge 
use. Quality of life was chosen as a more holistic and 
patient-centered measure of impact of knowledge use. 

HbAlc, systolic and diastolic blood pressure, LDL-C 
and weight to be collected every 3 months; these out- 
comes were chosen to help inform sample size calcula- 
tions of future prospective trials. 

Data collection 

Data on outcomes will be collected by patient-completed 
questionnaires; patients will be compensated with a small 
honorarium for each questionnaire. For the pre- 
implementation phase, aggregates of patient data will be 
obtained every three weeks for nine months, resulting in 
12 data points. The intervention will then be implemen- 
ted. For the post-implementation phase, aggregates of pa- 
tient data will be obtained every three weeks for nine 
months following the intervention, resulting in another 12 
data points. HbAlc and LDL-C will be collected from the 
hospital medical records. Systolic and diastolic blood pres- 
sure will be collected by the research coordinator with an 
Omron Blood Pressure Model (Model no. HEM-907XL) 
and recorded as the average of 3 readings. Weight will be 
collected by the research coordinator with a Seca digital 
scale (Model 707). Web server log analysis will be done to 
assess frequency and duration of the specific components 
of the intervention; metrics selected are described in Ap- 
pendix C. All participants will be asked at the end of the 
study to disclose whether other web-based interventions 



Table 2 Description of primary outcome scales 



Outcome 



Scale 



Description 



Self-efficacy 



Modified Grossman Self-efficacy The scale contains 25 items that measure the intensity of self-efficacy for activities of the diabetes 



for Diabetes Scale [36,37] 



Self-care 
behavior 



Summary of Diabetes Self-Care 
Activities Measure - Revised 
[38,39] 



Diabetes-specific 
quality of life 



Diabetes Distress Scale [40] 



regimen. Subjects are asked to describe how much they believe they could or could not do what 
was stated. The responses on this 6-point scale range from "very sure I can't" to "very sure I can" 
do what was stated in each item. Higher scores indicate greater confidence in one's ability to 
perform the designated treatment activities. The following statements are examples of the self- 
efficacy items: "Figure meals and snacks at home" and "Keep track of blood sugar levels". The 
modified self-efficacy scale has a moderate to high reliability (Cronbach's alpha = 0.51 to 0.86). 

Items selected from this self-report instrument assess participants' frequency (over the past 
7 days) of engaging in diabetes self-care behaviors, including following a healthy diet, spacing 
out carbohydrates evenly across the day, physical activity, self-monitoring of blood glucose 
testing, foot care, and medication and/or insulin taking. For each diabetes self-care behavior, 
participants are asked to respond using the following prompt: "On how many of the last 
7 days. . ." Responses, which are based on a 7-day week, range from 0 days to 7 days. Greater 
number of days indicated better self-management. Reliability and validity for this instrument have 
been found to be adequate, with a test-retest correlation of 0.40 and internal consistency of 0.47. 

The DDS is a 17 item instrument that assesses emotional distress and functioning specific to 
living with diabetes. Responses are scored on a 6-point Likert-type scale from 1 = "no problem" to 
6 = "serious problem". Scores can range from 17-102 with higher scores indicating poorer 
diabetes-related quality of life and lower scores indicating better diabetes-related quality of life. 
The DDS has been found to have high internal reliability with a Cronbach's alpha of 0.93, good 
convergent validity with the Center for Epidemiological Studies Depression Scale (CESD) (r = 0.56) 
and self-care behaviours including lower adherence to eating recommendations (r = 0.30, 
p < 0.001) and lower levels of physical activity (r = 0.1 3, p< 0.01) [40]. In addition, diabetes distress 
has been demonstrated to be associated with HbAlc (r = 0.1 7-0.31 , P = .00-.001), diet (r = -0.38, 
P = .00), physical activity (r = -0.1 3, P = .01) and medication adherence (r = -0.16, P = .00) [41,42]. 
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were used, and if so, whether they employed text- or 
image-based didactic materials, interactive technology or 
behavioral strategies. 

Sample size calculation 

Using a range of correlations from 0.2-0.8, a 0.05 signifi- 
cance level, and a power of 80%, a sample of at most 52 
subjects is required to detect a change of 0.5 units in 
self-efficacy score before the intervention as compared 
to after. A formula for paired mean comparisons was 
used [43] and the longitudinal nature of the study will 
only increase the power [44]. The sample size was fur- 
ther adjusted to account for a 40% expected dropout 
rate (attrition). 

Data analysis 

Descriptive analysis will examine the distribution of each 
variable of interest and determine which variables are suit- 
able as covariates for modeling. In addition, exploratory 
and graphical measures will be used for each of the pri- 
mary and secondary outcomes to determine the distribu- 
tion of the outcome of interest to enable a suitable 
statistical model to be constructed. Should the distribution 
of the outcome not be ideal, a suitable transformation will 
be used to enable proper statistical modeling [45,46]. The 
primary analysis, assessing the effect of the intervention 
on self-efficacy, will be examined using a linear mixed 
model to account for inherent correlation of the repeated 
measures within each individual [47]. The model will 
examine the effect of intervention on the outcome, self-ef- 
ficacy, while adjusting for the following variables: age, eth- 
nicity, education level, employment status, income (above 
or below $30 000) and health literacy. Should the data 
support it the model will also include interactions of these 
variables with the intervention to determine whether the 
impact of the intervention on self-efficacy varies for differ- 
ent levels of the variables. Time will also be included in 
the model to adjust for any trends over time. Model diag- 
nostics will be performed using residual analysis. 

Similar modeling will be used for the other outcomes 
of self-care and quality of life as well as A1C, systolic BP, 
diastolic BP, LDL and weight. For secondary outcomes, 
due to the data being collected less frequently, fewer 
variables will be included in the model. Each model in 
the secondary analysis will examine the effect of the 
intervention while adjusting for self-efficacy, income and 
ethnicity. In addition to these covariates, the outcomes 
systolic BP, diastolic BP and LDL will also include age as 
a covariate while the analysis examining weight as the 
dependent variable will also include insulin use as a cov- 
ariate. Should the data support it these variables will also 
be examined as interaction terms with the intervention. 

This analysis approach will allow us to examine the ef- 
fect of the intervention on each of the primary and 



secondary outcomes while adjusting for other important 
variables and accounting for the correlation that arises 
in repeated measures data. 

Other variables of interest such as website usage statis- 
tics, website user satisfaction and subscales of self-effi- 
cacy, self-care and quality of life will be examined using 
descriptive and graphical analysis. 

In-depth interviews 

Following completion of the repeated measures study, 
in-depth focused, individual interviews will be used to 
assess acceptability, usability, strengths and weaknesses 
of the intervention, facilitators and barriers to its use, 
user satisfaction, and sustainability of use. 

Participants 

A purposive sample of participants (based on gender, 
age, ethnicity, duration of diabetes, educational attain- 
ment, annual income, pre-intervention experience with 
website use) from the repeated measures study will be 
recruited by the study coordinator. A sample size of 25 
participants is likely to be necessary to achieve theoret- 
ical saturation. 

Outcomes 

The perceived impact of each web tool on participants' 
knowledge of diabetes self-management, perceptions 
regarding their experiences of diabetes and how it 
impacts their lives, stage of behavior change, perceived 
self-efficacy and self-management will be assessed, 
prompted by open-ended questions from the inter- 
viewer. Participants' perspectives regarding barriers and 
facilitators to the adoption and sustained use of a web- 
based intervention, types of information included and 
formats used (and preferred) will be discussed. Accounts 
of their use of other health information resources, their 
interaction with the healthcare system, and their percep- 
tions of online privacy and sharing will also be explored. 

Data collection and analysis 

All interviews will be audio tapes and transcribed verba- 
tim [30] . Transcripts will be coded for emergent categories 
and themes using constant comparison [31]. Transcripts 
will be coded and reviewed independently by three team 
members with expertise in qualitative research methods. 
A coding framework will be developed based on the emer- 
ging analysis, and then tested and refined with subsequent 
interviews in an iterative process. The coding framework 
will be based on consensus amongst the analysts [34,35]. 
NVivo software (version 9) will be used to assist with data 
management and retrieval for analysis. In addition, the 
qualitative data will be compared with the quantitative 
data in order to triangulate and interpret both datasets in 
the context of the whole study [48]. 
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Research ethics 

The study was approved by the Research Ethics Board of 
St. Michael's Hospital (reference number 09-091) and 
Sunnybrook Health Sciences Centre (reference number 
177-2009). 

Discussion 

Principal results 

While the first two phases of this study will shed light 
on information needs of patients with type 2 diabetes, 
our subsequent phases will assess the impact of this 
systematically developed intervention. Specifically, the 
repeated measures design will determine the impact of 
web-based self-management programs on important 
psychological and clinical outcomes. We will also iden- 
tify factors (both of participants and of the website) that 
correlate with website use and effectiveness in order to 
guide future intervention development and research. 

Strengths and limitations 

Strengths of our approach include our rigorous theory- 
driven and evidence-based approach to intervention de- 
velopment, our systematic refinement of the interven- 
tion based on feasibility and usability data, our use of 
mixed methods for evaluation, and our selection of us- 
ability assessment techniques. The importance of theory 
was highlighted in a recent systematic review which 
found that effectiveness of internet-based interventions 
is associated with more extensive use of theory [17]. 
Similarly, evidence-based content should form the foun- 
dation of an educational website, yet pre-existing dia- 
betes websites have wide variations in the quality of 
evidence leveraged [11]. Finally, stakeholder input and 
usability testing are key to intervention implementation 
and sustainability [49] but are frequently neglected, with 
up to 60% of diabetes-related websites having at least 
three usability errors [13]. These elements of design - 
theory, evidence, stakeholder input, and usability testing 
- are fundamental to intervention effectiveness and 
should represent the gold standard for intervention de- 
velopment [13]. 

Within the qualitative evaluation, we will use trained 
moderators who are not otherwise invested in the project, 
collect background information regarding participants to 
ensure that there is diverse representation, and use a 
sound methodology for coding and interpreting the data 
into relevant themes, including independent coding by 
two individuals and interpretation by three individuals to 
ensure data trustworthiness [50] . In addition, triangulation 
of the qualitative findings with the quantitative results will 
promote methodological rigor [48]. One limitation of our 
usability testing approach is that this technique may miss 
consistency problems (for example, conventions, such as 
an arrow indicating "next", not being used consistently 



throughout the website). However, we will address this 
issue by ensuring that our sessions are moderated and 
analyzed by a usability expert. 

Potential limitations of our evaluation study include the 
consistency with which participants will respond at each 
time point, generalizability, as well as maturation and his- 
tory threats to internal validity. Response rate will be max- 
imized with the use of reminder emails and honoraria. We 
will maximize generalizability by recruiting broadly across 
all socio-demographic strata and by collecting data to 
allow us to characterize our study populations. We will ac- 
count for maturation threats by assessing for interaction 
with time, and we will assess for history threats by record- 
ing secular events that could impact on our outcomes 
prospectively throughout the study period and by asking 
our participants on their exit questionnaires. 

Conclusions 

Web-based interventions to improve self-management 
of diabetes show great promise because they target pa- 
tient behaviors directly, can be easily scaled up given the 
fixed cost of website development and are inexpensive 
to maintain. This study builds on previous work using 
the most rigorous approach to intervention develop- 
ment, including usability testing, qualitative studies of 
acceptability, iterative refinement, and measures the im- 
pact on self-efficacy, quality of life and risk factor con- 
trol using a repeated measures design. Our intervention 
is based robustly on a theoretical foundation of self-effi- 
cacy, which has been demonstrated to predict and pro- 
mote behavior change and improve health outcomes in 
chronic diseases such as diabetes mellitus. In addition, 
use of behavior change theory itself in intervention de- 
velopment is associated with greater intervention effect- 
iveness. This, in combination with evidence-based 
content and strategies, stakeholder input and usability 
testing, should result in an optimally engineered website 
designed to improve psychological and clinical outcomes 
and complement health care delivery. Increasing use of 
the World Wide Web by consumers for health informa- 
tion and ongoing revolutions in social media are strong 
indicators that consumers are primed to welcome a new 
era of technology in health care. However, their full po- 
tential is hindered by limited knowledge regarding their 
effectiveness, high prevalence of usability errors, and 
high attrition rates. Our development and research 
agenda aims to address these limitations by assessing ef- 
fectiveness, addressing usability errors, identifying char- 
acteristics associated with website use and attrition, and 
developing strategies to reduce website attrition in order 
to maximize clinical outcomes. 
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